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NOTICE:  When  goverument  or  other  ttravings,  speci¬ 
fications  or  other  data  are  used  for  any  purpose 
other  chan  in  connection  vith  a  definitely  related 
government  procurement  operation,  the  U.  S. 
Government  thereby  Incurs  no  reBponsibllity,  nor  any 
obligation  whatsoever;  and  the  fact  that  the  Govern¬ 
ment  may  have  formulated,  furnished,  or  in  any  way 
supplied  the  said  drawings,  specifications,  or  other 
data  Is  not  to  be  regarded  by  implication  or  other¬ 
wise  ac  in  any  manner  licencing  the  holder  or  any 
other  person  or  corporation,  or  conveying  any  rights 
or  permission  to  manufacture,  use  or  sell  any 
patented  invention  that  may  in  any  way  be  related 
thereto. 


U,  S.  NAVAL  SCHOOL  OF  AVIATION  MEDICINE 
U.  S.  NAVAL  AVIATION  MEDICAL  CENTER 
PENSACOLA,  FLORIDA 


0^ 

Od 

CV) 


o 

,c:a 

■v^iJ 

C'.> 

O 


THE  VALIDITY  OF  THE  OFFICER-LIKE-QUALITY  MEASURES 
USED  IN  THE  U.  S.  NAVAL  SCHOOL,  PRE-FLIGHT 


SPECIAL  REPORT  No.  61-6 

hL 

Lieut-enant  (jg)  R.  E.  Doll,  MSC,  USNR 
and 


J.  R.  Berkshire 


Approved  by 


Capt-nin  Ashfon  Graybiel,  MC,  USN 
Director  of  Research 


XlvlUTX 


Released  by 

Captain  Clifford  P.  Phoebus,  MC,  USN 
Commanding  Officer 


20  JUNE  1961 


Opinions  or  conclusions  contained  in  this  report  are  those  of  the  authors.  They 
are  not  to  be  construed  as  necessarily  reflecting  the  viewer  the  endorsement  of 
the  Navy  Department.  Reference  may  be  made  to  this  report  in  the  same  way 
os  to  publisheJ  articles  noting  author,  title,  source,  date,  and  report  number. 


SUMMARY  PAGE 


THE  PROBLEM 

The  purpofe  of  this  investigation  was  to  determine  the  relative  validities 
of  the  components  of  the  pre-flight  officer-like-quality  grade  (OLQ)  and  the 
weights  which,  when  applied  to  these  components,  would  yield  the  most  valid 
composite  grade, 

flNDSNGS 

The  current  procedure  of  using  plus  and  minus  factors  for  cadet  rank, 
activities,  and  demerits  makes  the  OLQ  grade  less  accurate  as  a  predictor  of 
subsequent  attr  t’on,  Weightings  of  ,4.  «2,  .4  for  average  peer  rating,,  average 
instructor  lat'ug,  and  overage  military  rating ,  respectively,  would  increase  the 
validity  of  the  OLQ  grade. 

RECOMMENDATIONS 

The  aveiage  OLQ  grade  should  not  be  modified  by  addition  or  subtraction 
foi  cadet  '  an|< ,  activities, and  demerits. 

No  char-ge  n'l  weights  should  be  made  at  this  time,  pending  more  detailed 
'■.luu'ies  of  the  coiriros>ioir  of  the  overage  instructor  rating  and  the  average  military 
I  a  I  uic) . 


iNTRODUCTION 


A  feature  of  the  quality  control  procedures  current  in  nova!  aviation  training 
is  the  use  of  grades  already  mode  by  a  student  to  assist  administrators  in  deciding 
whether  a  failing  student  should  be  dropped  or  given  another  chance.  The  objectlva 
of  this  procedure  is  the  early  elimination  from  training  of  students  who  hove  a  high 
probability  of  later  failure.  Information  concerning  the  probabilities  of  failure 
related  to  various  grades  already  made  by  the  student  is  provided  in  CNATRA 
Instruction  1610. 5A  (1).  One  of  the  more  dependable  of  these  reference  grades  as 
a  predictor  of  subsequent  success  in  training  is  the  pre“flight  final  over-all  grade. 
This  grade  is  a  weighted  composite  of  all  of  the  grades  made  in  the  U.S.  Naval 
School,  Pre“Flight,  the  weights  having  been  determined  so  as  to  ^ive  the  most 
accurate  possible  prediction  of  subsequent  success. 

For  naval  aviation  cadets  and  aviation  officer  candidates  one  of  the  major 
components  of  this  weighted  grade  is  the  officer-like-quality  rating  (OLQ)  which 
takes  30  per  cent  of  the  total  weight.  The  OLQ  grade,  however,  is  itself  a  com¬ 
posite,  and  while  the  weight  that  it  should  take  in  the  pre-flight  grade  has  been 
determined  by  statistical  processes,  the  optimum  weights  of  the  ingredients  of  the 
OLQ  grade  have  not  been  so  determined. 

This  study  examines  the  relative  validities  of  the  components  of  the  pre- 
flight  OLQ  grade  and  determines  the  weights  which  should  be  applied  to  each  to 
yield  the  most  valid  composite. 

THE  OFFICER-LIKE-QUALITY  GRADE 

The  OLQ  grade  is  a  composite  of  the  following  three  sub-grades,  each  of 
which  is  also  a  composite; 

A.  Average  Peer-Rating 

In  the  eighth  week  of  training,  NavCad's  and  AOC’s  are  asked  to  nominate 
from  their  class  the  three  top  men  and  the  three  bottom  men  in  terms  of  their  leader¬ 
ship  potential.  These  nominations  are  totaled  and  the  totals  converted  to  standard 
scores.  In  the  twelfth  week  the  process  is  repeated.  The  first  component  of  the  OLQ 
grade  is  the  average  of  these  two  ratings  with  a  35  per  cent  weight, 

B.  Average  Instructor  Rating 

« 

At  the  time  that  data  for  this  study  were  collected  (1958)  the  instructor  sub¬ 
grade  was  the  average  of  ratings  assigned  to  students  by  their  Instructors  in  Aviation 
Science,  Leadership,  and  Survival.  In  each  of  these  the  instructor  named  the  top 
three  and  bottom  three  men  in  the  class  in  terms  of  his  judgment  of  their  leadership 
potential.  The  highest  nominee  received  a  score  of  86,  the  next  74,  the  third  62; 
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the  bottom  man  14,  next  bottom  26,  and  third  from  bottom  38;  all  unrated  men 
received  50,  Beginning  in  1959  ratings  by  the  military  tactical  officer  were 
substituted  for  those  of  the  survival  instructor.  The  tactical  officer  assigns  a 
yi  ude  each  week  which  represents  his  opinion  of  the  man's  leadership  behavior 
during  that  week.  At  the  end  of  fourteen  weeks  these  weekly  grades  are 
averaged..  The  result  is  combined  wirti  the  aviation  science  and  the  leadership 
instructors'  ratings  with  weights  of  40,  30,  30,  respectively.  This  instructor 
rating  then  becomes  35  per  cent  of  the  OLQ  grade. 

C,  Military  Rating 

The  ihltd  pcvtion  of  tfie  OLQ  grade  is  the  military  rating.  Forty  per 
cent  of  this  grade  is  assigned  by  the  drill  sergeant,  based  upon  the  man's  per¬ 
formance  in  military  drill;  40  per  cent  is  assigned  by  the  tactical  officer,  based 
upon  how  well  the  man  passed  inspections;  and  20  per  cent  is  awarded  by  the 
f.cidet  officer  of  the  battalion  in  respect  to  how  well  the  man  stands  his  assigned 
watches.  Tliis  weighted  military  grade  is  given  a  weight  of  30  per  cent  in  the 
OLQ  composite,.  The  composite  is  then  modified  by  subtracting  points  for  fhe 
('umber  of  demerits  the  man  received,  and  adding  points  for  his  rank  as  a  cadet 
officer  ond  for  extra-curricular  activities,  such  as  singing  in  the  choir,  ushering 
(it  cliapel,  or  committee  work, 

PROCEDURES  AND  RESULTS 

On  p<'g«  29  of  CNATRA  Manual  P-69  (3)  the  point  is  marJe  that,  if  a 
student's  grades  are  to  be  used  or  useful  in  making  administrative  decisions  about 
rfiul  iiudetn,  it  is  wortirwhiie  to  weight  grades  so  that  they  give  the  most  accurate 
predic.tion  of  later  success  or  failure.  With  this  in  mind,  and  by  use  of  data 
cbtoined  fiotn  the  records  of  951  cadets  who  completed  pi-e-flight  during  1958, 
the  inter-relationships  of  the  OLQ  components  and  training  success  criteria  were 
determined ,  ■-  “ 


THE  criteria 


Cases  that  did  not  complete  the  flight  training  program  were  divided  into 

tliree  types; 


LM 


Those  who  voluntarily  withdrew. 


FAIL  -  Those  who  were  dropped  from  the  program  because  of  failure 
to  meet  minimum  standards  in  flight,  or  academic,  or  officer 
performance. 


OTHER  “  Those  who  left  the  program  for  reasons,  mainly  physical,  other 
than  LM  or  FAIL, 
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A  fourth  criterion  grouping  in  this  study  is  the  total  (TOT)  of  these  three  types 
of  separations  from  training. 

ANALYSIS 

Table  I  shows  the  intercorrelations  of  the  OLQ  components  and  composites 
and  the  bi-serial  correlations  of  each  component  and  composite  with  the  four  types 
of  dichotomous  completion-elimination  criteria. 

Note  first  that,  as  shown  in  the  table,  the  average  peer  rating  has  a  slightly 
higher  validity  than  does  either  the  8th  or  12th  week  peer  ratings  alone  for  the  LM, 
FAIL,  and  TOT  criteria.  The  average  instructor  rating  has  very  little  validity.  The 
average  military  rating  has  nearly  as  high  validities  as  does  the  average  peer  rating. 

The  average  of  the  three  has  a  slightly  higher  correlation  with  FAIL,  OTHER,  and 
TOT  than  does  the  average  peer  rating  along,  but  is  slightly  poorer  as  a  predictor 
of  LM  cases.  The  OLQ  grade  (which  is  variable  6  modified  by  a  minus  factor  for 
demerits  and  plus  factors  for  cadet  rank  and  extra-curricular  activities)  is  less  valid 
than  the  unmodified  average;  in  other  words,  the  minus  and  plus  factors  ,  'ake  the 
grade  worse,  lather  than  better. 

Inspection  of  the  standard  deviations  (S.D.)  shows  substantial  differences  in 
the  S.D.  's  of  the  averages  which  make  up  the  OLQ  (variables  3,  4,  and  5).  Because 
of  these  differences  the  real  weights  of  the  components  into  the  OLQ  are  approximately 
31,  24,  14  instead  of  the  35,  35,  30  intended. 

OPTIMUM  WEIGHTS 

In  order  to  dotorminc  how  the  components  should  ba  weighted  so  that  the 
composite  would  most  accurately  predict  subser^uent  failure  (FAIL),  multiple  correlation 
procedures  were  applied  to  the  data  in  Table  I.  The  result  is  limited  by  the  fact  that 
the  combinations  of  sub-grades  within  the  "Instructor  rating"  and  the  "Military  rating" 
aro  almost  certainly  not  optimal,  and  by  the  possibility  that  the  small  criterion 
correlations  of  the  "Instructor  rating"  may  represent  academic  ability — which  is  better 
measured  by  academic  grades.  Within  these  limits,  it  can  be  said  that  the  correlation 
of  variable  6  with  the  failure  criterion  is  raised  from  the  .37  shown  in  Table  3  to  .42 
when  the  components  ate  weighted  as  follovrs: 

Av.  Peer  rating  .4 

Av.  Inst,  rating  .2 

Av.  Mil.  rating  .4 
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CONCLUSIONS  AND  RECOMMENDATIONS 


The  fact  that  the  correlations  of  the  final  pre-flight  OLQ  grade  with  the 
various  types  of  attrition  are  lower  than  those  of  the  average  OLQ  grade  before 
the  application  of  plus  and  minus  factors  for  cadet  rank,  activities,  and  demerits 
indicates  that  these  modifications  are  making  the  grade  less  accurate  as  a  predictor 
of  subsequent  attrition.  These  procedures  should  be  discontinued. 

Although  this  study  indicates  that  a  substantial  increase  in  the  validity  of 
the  OLQ  grade  would  result  from  a  change  in  the  weighting  of  the  components,  it 
is  recommended  thot  changes  be  deferred  until  studies  of  the  instructor  rating  and 
military  rating  components  can  be  conducted.  It  is  almost  certain  that  such  studies 
will  result  in  further  recommendations  for  change  and  in  still  more  improvement  in 
validity . 


REFERLNCifS 

1  ,  A  Guide  for  Pr<!diction  of  Student  Success.  CNATRA  INST  1610,  5A. 
15  Apr  I960. 

,  fiork'jhii  c; ,  j.K.,  I  liiniun -..lurdity  contiol  in  nava)  nil  tiuining-  1960, 
NavnI  Rc'scurc.b  Reviews ,  lumKii  y ,  p  1961. 

Tt,  MeasuriiK]  I'l uitiiiig  Progress.  C.iNAVRA  P- 69,  Pensucoio,  FOu  . 
Naval  Ail  liuiriiny  Commend,  I960. 


5 


UNCLASSIFIED 


UNCLASSIFIED 


